Use Keras experiment to transfer style Watson Machine Learning icon
Icon

This notebook contains the steps and code required to demonstrate style transfer techique using Watson Machine Learning Service. This notebook introduces commands for getting data, training_definition persistance to Watson Machine Learning repository and model training.

Some familiarity with Python is helpful. This notebook uses Python 3.6 and Watson Studio environments.

Learning goals

In this notebook you learn to work with Watson Machine Learning experiments to train Deep Learning models (Keras).

Contents

  1. Set up
  2. Create the training definition
  3. Define the experiment
  4. Run the experiment
  5. Results
  6. Summary

1. Set up

Before you use the sample code in this notebook, you must perform the following setup tasks:

  • Create a Watson Machine Learning (WML) Service instance (a free plan is offered and information about how to create the instance is here).
  • Create a Cloud Object Storage (COS) instance (a lite plan is offered and information about how to order storage is here).
    Note: When using Watson Studio, you already have a COS instance associated with the project you are running the notebook in.
  • Create new credentials with HMAC:

    • Go to your COS dashboard.
    • In the Service credentials tab, click New Credential+.
    • Add the inline configuration parameter: {"HMAC":true}, click Add. (For more information, see HMAC.)

      This configuration parameter adds the following section to the instance credentials, (for use later in this notebook):

      "cos_hmac_keys": {
            "access_key_id": "-------",
            "secret_access_key": "-------"
       }

In this section:

1.1 Work with Cloud Object Storage (COS)

Import the Boto library, which allows Python developers to manage COS.

In [ ]:
!pip install --upgrade keras==2.2.4
In [2]:
# Import the boto library
import ibm_boto3
from ibm_botocore.client import Config
import os
import json
import warnings
import urllib
import time

Authenticate to COS and define the endpoint you will use.

  1. Enter your COS credentials in the following cell. You can find these credentials in your COS instance dashboard under the Service credentials tab as described in the set up section.

  2. Go to the Endpoint tab in the COS instance's dashboard to get the endpoint information, for example: s3-api.us-geo.objectstorage.softlayer.net.

In [3]:
# Enter your COS credentials.
cos_credentials = {
  "apikey": "***",
  "cos_hmac_keys": {
    "access_key_id": "***",
    "secret_access_key": "***"
  },
  "endpoints": "https://cos-service.bluemix.net/endpoints",
  "iam_apikey_description": "***",
  "iam_apikey_name": "***",
  "iam_role_crn": "crn:v1:bluemix:public:iam::::serviceRole:Writer",
  "iam_serviceid_crn": "***",
  "resource_instance_id": "***"
}
In [5]:
api_key = cos_credentials['apikey']
service_instance_id = cos_credentials['resource_instance_id']
auth_endpoint = 'https://iam.bluemix.net/oidc/token'
# Enter your Endpoint information.
service_endpoint = 'https://s3-api.us-geo.objectstorage.softlayer.net'

Create the Boto resource by providing type, endpoint_url and credentials.

In [6]:
cos = ibm_boto3.resource('s3',
                         ibm_api_key_id=api_key,
                         ibm_service_instance_id=service_instance_id,
                         ibm_auth_endpoint=auth_endpoint,
                         config=Config(signature_version='oauth'),
                         endpoint_url=service_endpoint)

Create the buckets you will use to store training data and training results.

Note: Bucket names must be unique.

In [7]:
# Create two buckets, style-data-example and style-results-example
from uuid import uuid4

bucket_uid = str(uuid4())

buckets = ['style-data-example-' + bucket_uid, 'style-results-example-' + bucket_uid]
for bucket in buckets:
    if not cos.Bucket(bucket) in cos.buckets.all():
        print('Creating bucket "{}"...'.format(bucket))
        try:
            cos.create_bucket(Bucket=bucket)
        except ibm_boto3.exceptions.ibm_botocore.client.ClientError as e:
            print('Error: {}.'.format(e.response['Error']['Message']))
Creating bucket "style-data-example-4abd3320-d400-4c94-89f5-2b7099c9c16c"...
Creating bucket "style-results-example-4abd3320-d400-4c94-89f5-2b7099c9c16c"...

You have now created two new buckets:

  • style-data-example
  • style-results-example

Display a list of buckets for your COS instance to verify that the buckets were created.

In [8]:
# Display the buckets
print(list(cos.buckets.all()))
[s3.Bucket(name='1021-donotdelete-pr-1g4y74kpycwgpq'), s3.Bucket(name='ananyadoc-donotdelete-pr-e8qrksvnxfjml5'), s3.Bucket(name='ananyafirst-donotdelete-pr-qwlzno4acd5flp'), s3.Bucket(name='ananyatest-donotdelete-pr-vmaoyh1xzi3oxl'), s3.Bucket(name='canvasmanualtestproject-donotdelete-pr-bfdvhe8rlbopoi'), s3.Bucket(name='style-data-example-479106e2-8fad-4205-a8f7-d8b08e721a9a'), s3.Bucket(name='style-data-example-4abd3320-d400-4c94-89f5-2b7099c9c16c'), s3.Bucket(name='style-data-example-ac626a53-42ef-4393-8dda-bc8de9968484'), s3.Bucket(name='style-data-example-b3223efb-0e68-472a-91d8-d01c63db4747'), s3.Bucket(name='style-results-example-479106e2-8fad-4205-a8f7-d8b08e721a9a'), s3.Bucket(name='style-results-example-4abd3320-d400-4c94-89f5-2b7099c9c16c'), s3.Bucket(name='style-results-example-ac626a53-42ef-4393-8dda-bc8de9968484'), s3.Bucket(name='style-results-example-b3223efb-0e68-472a-91d8-d01c63db4747'), s3.Bucket(name='tensorflow-world'), s3.Bucket(name='training-data-54fca992-9e0a-4b5b-8c88-211c8487fe80'), s3.Bucket(name='training-results-54fca992-9e0a-4b5b-8c88-211c8487fe80')]

1.2 Download training data and upload it to COS buckets

Download your training data and upload them to the 'training-data' bucket. Then, create a list of links for the training dataset.

The following code snippet creates the STYLE_DATA folder and downloads the files from the links to the folder.

Tip: First, use the !pip install wget command to install the wget library:

In [ ]:
!pip install --upgrade wget
In [ ]:
import wget, os

# Create folder
data_dir = 'STYLE_DATA'
if not os.path.isdir(data_dir):
    os.mkdir(data_dir)

links = ['https://github.com/fchollet/deep-learning-models/releases/download/v0.1/vgg19_weights_tf_dim_ordering_tf_kernels_notop.h5',
         'https://upload.wikimedia.org/wikipedia/commons/thumb/e/ea/Van_Gogh_-_Starry_Night_-_Google_Art_Project.jpg/1513px-Van_Gogh_-_Starry_Night_-_Google_Art_Project.jpg',
         'https://upload.wikimedia.org/wikipedia/commons/5/52/Krak%C3%B3w_239a.jpg',
         'https://upload.wikimedia.org/wikipedia/commons/3/3f/Kandinsky%2C_Lyrisches.jpg']

# Download the links to the folder
for i in range(len(links)):
    if 'Gogh' in links[i]: 
        filepath = os.path.join(data_dir, 'van_gogh.jpg')
    elif 'Krak' in links[i]: 
        filepath = os.path.join(data_dir, 'krakow.jpg')
    elif 'Kandinsky' in links[i]:
        filepath = os.path.join(data_dir, 'kandinsky.jpg')
    else:
        filepath = os.path.join(data_dir, os.path.join(links[i].split('/')[-1]))

    if not os.path.isfile(filepath):
        print(links[i])
        urllib.request.urlretrieve(links[i], filepath)

# List the files in the STYLE_DATA folder        
!ls STYLE_DATA

Base image: Cracow - main market square

In [11]:
from IPython.display import Image
Image(filename=os.path.join(data_dir, 'krakow.jpg'), width=1000)
Out[11]:

Style image 1: Vincent Van Gogh - Starry Night

In [12]:
Image(filename=os.path.join(data_dir, 'van_gogh.jpg'), width=500)
Out[12]:

Style image 2: Kandinsky Lyrisches

In [13]:
Image(filename=os.path.join(data_dir, 'kandinsky.jpg'), width=600)
Out[13]:

Upload the data files to the created buckets.

In [14]:
bucket_name = buckets[0]
bucket_obj = cos.Bucket(bucket_name)
In [15]:
for filename in os.listdir(data_dir):
    with open(os.path.join(data_dir, filename), 'rb') as data: 
        bucket_obj.upload_file(os.path.join(data_dir, filename), filename)
        print('{} is uploaded.'.format(filename))
van_gogh.jpg is uploaded.
krakow.jpg is uploaded.
vgg19_weights_tf_dim_ordering_tf_kernels_notop.h5 is uploaded.
kandinsky.jpg is uploaded.

Let's see the list of all the buckets and their contents.

In [16]:
for obj in bucket_obj.objects.all():
    print('Object key: {}'.format(obj.key))
    print('Object size (kb): {}'.format(obj.size/1024))
Object key: kandinsky.jpg
Object size (kb): 337.97265625
Object key: krakow.jpg
Object size (kb): 2063.50390625
Object key: van_gogh.jpg
Object size (kb): 833.9755859375
Object key: vgg19_weights_tf_dim_ordering_tf_kernels_notop.h5
Object size (kb): 78256.46875

You are done with COS, and you are now ready to train your model!

1.3 Work with the Watson Machine Learning instance

Load the libraries you need.

In [17]:
import urllib3, requests, json, base64, time, os

Authenticate to the Watson Machine Learning (WML) service on IBM Cloud.

Tip: Authentication information (your credentials) can be found in the Service credentials tab of the service instance that you created on IBM Cloud. If there are no credentials listed for your instance in Service credentials, click New credential (+) and enter the information required to generate new authentication information.

Action: Enter your WML service instance credentials here.

In [18]:
wml_credentials = {
    "apikey": "***",
    "instance_id": "***",
    "password": "***",
    "url": "https://ibm-watson-ml.mybluemix.net",
    "username": "***"
}

Install the watson-machine-learning-client from pypi.

In [ ]:
!pip install --upgrade watson-machine-learning-client

Import the watson-machine-learning-client and authenticate to the service instance.

In [21]:
from watson_machine_learning_client import WatsonMachineLearningAPIClient
In [22]:
client = WatsonMachineLearningAPIClient(wml_credentials)
In [23]:
print(client.version)
1.0.376

Note: watson-machine-learning-client documentation can be found here.

2. Create the training definitions

2.1 Prepare the training definition metadata

Hint: The final effect depends on number of iterations, and that the number of iterations impacts the training time.

In [24]:
#Set the number of iterations.
iters = 1
In [25]:
model_definition_1_metadata = {
            client.repository.DefinitionMetaNames.NAME: "style transfer van gogh",
            client.repository.DefinitionMetaNames.FRAMEWORK_NAME: "tensorflow",
            client.repository.DefinitionMetaNames.FRAMEWORK_VERSION: "1.14",
            client.repository.DefinitionMetaNames.RUNTIME_NAME: "python",
            client.repository.DefinitionMetaNames.RUNTIME_VERSION: "3.6",
            client.repository.DefinitionMetaNames.EXECUTION_COMMAND: "python style_transfer.py krakow.jpg van_gogh.jpg krakow --iter " + str(iters)
            }
In [26]:
model_definition_2_metadata = {
            client.repository.DefinitionMetaNames.NAME: "style transfer kandinsky",
            client.repository.DefinitionMetaNames.FRAMEWORK_NAME: "tensorflow",
            client.repository.DefinitionMetaNames.FRAMEWORK_VERSION: "1.14",
            client.repository.DefinitionMetaNames.RUNTIME_NAME: "python",
            client.repository.DefinitionMetaNames.RUNTIME_VERSION: "3.6",
            client.repository.DefinitionMetaNames.EXECUTION_COMMAND: "python style_transfer.py krakow.jpg kandinsky.jpg krakow --iter " + str(iters)
            }

2.2 Get the sample model definition content files from Git

In [27]:
!rm -rf STYLE.zip
In [ ]:
filename_definition = 'STYLE.zip'

if not os.path.isfile(filename_definition):
    !wget https://github.com/pmservice/wml-sample-models/raw/master/keras/style/definition/STYLE.zip

!ls STYLE.zip

2.3 Store the training definition in the WML repository

Store definition 1

In [29]:
definition_details = client.repository.store_definition(filename_definition, model_definition_1_metadata)

definition_url = client.repository.get_definition_url(definition_details)
definition_uid = client.repository.get_definition_uid(definition_details)
print(definition_url)
https://us-south.ml.cloud.ibm.com/v3/ml_assets/training_definitions/b9bd502b-7665-44ff-861e-db686c528fa2

Store definition 2

In [30]:
definition_2_details = client.repository.store_definition(filename_definition, model_definition_2_metadata)

definition_2_url = client.repository.get_definition_url(definition_2_details)
definition_2_uid = client.repository.get_definition_uid(definition_2_details)
print(definition_2_url)
https://us-south.ml.cloud.ibm.com/v3/ml_assets/training_definitions/9715bf2e-fe48-486d-be72-d77f6ae94809

List the stored definitions

In [31]:
client.repository.list_definitions()
------------------------------------  -------------------------------  ------------------------  ------------------
GUID                                  NAME                             CREATED                   FRAMEWORK
9715bf2e-fe48-486d-be72-d77f6ae94809  style transfer kandinsky         2019-11-11T22:04:29.670Z  tensorflow
b9bd502b-7665-44ff-861e-db686c528fa2  style transfer van gogh          2019-11-11T22:04:27.321Z  tensorflow
d7d50440-aab0-4c20-9eca-63157263b471  cycle_gan_training               2019-11-07T21:23:32.844Z  tensorflow
fa193aa6-7437-4752-a27e-39e858bb7f25  cycle_gan_training               2019-11-07T21:20:44.355Z  tensorflow
ed7a5c37-89b5-42b8-b9c2-244f654f0ae2  cycle_gan_training               2019-11-07T21:18:18.131Z  tensorflow
ccbab726-a17c-4c6c-a2a7-f7a25f898f6d  cycle_gan_training               2019-11-07T21:15:29.598Z  tensorflow
1074e2e8-79d3-4369-afea-df93d6b6d79b  cycle_gan_training               2019-11-07T21:07:11.030Z  tensorflow
a17caf43-9eb3-43e2-b9a4-d8a2b74c7e5b  gan_training                     2019-10-21T12:37:11.706Z  tensorflow
8c86777d-9543-4605-9fb5-882c371de20c  My definition name               2019-07-15T22:40:27.557Z  tensorflow
0acb4e2f-57d4-4278-be32-49255107fdd2  python-client-tutorial_training  2019-03-15T18:20:47.034Z  tensorflow
4a2d1c30-7db1-4874-8bf2-3e9db5bba975  MNIST-CNN                        2019-03-07T13:32:56.180Z  tensorflow
5f549d43-486d-4e75-a56a-252136c0a2e7  MNIST-MLP                        2019-03-07T13:32:54.458Z  tensorflow
85529f7f-8502-4f3c-a779-750e145afe47  Tensorflow - ddl                 2019-03-07T13:24:33.546Z  tensorflow-ddl
862c5253-b0c6-4a81-b473-776a5b09dd1b  Tensorflow - horovod             2019-03-07T13:24:31.841Z  tensorflow-horovod
4f4b08bb-ad9b-4ffb-ad5b-a64b96fe6244  Tensorflow - distributed         2019-03-07T13:24:30.171Z  tensorflow
607e8219-48ca-4546-9e5f-482c4c502433  My definition name               2019-03-06T12:17:53.957Z  tensorflow
------------------------------------  -------------------------------  ------------------------  ------------------

3. Create the experiment definition

Get a list of supported configuration parameters.

In [32]:
client.repository.ExperimentMetaNames.show()
--------------------------  ----  --------
META_PROP NAME              TYPE  REQUIRED
NAME                        str   Y
DESCRIPTION                 str   N
TAGS                        list  N
AUTHOR_NAME                 str   N
EVALUATION_METHOD           str   N
EVALUATION_METRICS          list  N
TRAINING_REFERENCES         list  Y
TRAINING_DATA_REFERENCE     dict  Y
TRAINING_RESULTS_REFERENCE  dict  Y
--------------------------  ----  --------

Create an experiment, which will train two models based on previously stored definitions.

In [33]:
TRAINING_DATA_REFERENCE = {
                            "connection": {
                                "endpoint_url": service_endpoint,
                                "access_key_id": cos_credentials['cos_hmac_keys']['access_key_id'],
                                "secret_access_key": cos_credentials['cos_hmac_keys']['secret_access_key']
                            },
                            "source": {
                                "bucket": buckets[0],
                            },
                            "type": "s3"
                        }
In [34]:
TRAINING_RESULTS_REFERENCE = {
                            "connection": {
                                "endpoint_url": service_endpoint,
                                "access_key_id": cos_credentials['cos_hmac_keys']['access_key_id'],
                                "secret_access_key": cos_credentials['cos_hmac_keys']['secret_access_key']
                            },
                            "target": {
                                "bucket": buckets[1],
                            },
                            "type": "s3"
                        }
In [35]:
experiment_metadata = {
            client.repository.ExperimentMetaNames.NAME: "STYLE experiment",
            client.repository.ExperimentMetaNames.TRAINING_DATA_REFERENCE: TRAINING_DATA_REFERENCE,
            client.repository.ExperimentMetaNames.TRAINING_RESULTS_REFERENCE: TRAINING_RESULTS_REFERENCE,
            client.repository.ExperimentMetaNames.TRAINING_REFERENCES: [
                        {
                            "name": "van gogh - cracow",
                            "training_definition_url": definition_url,
                            "compute_configuration": {"name": "k80x4"}
                        },
                        {
                            "name": "kandinsky - cracow",
                            "training_definition_url": definition_2_url,
                            "compute_configuration": {"name": "k80x4"}
                        },
                    ],
                }

Store the experiment in the WML repository.

In [36]:
# Store the experiment and display the experiment_uid.
experiment_details = client.repository.store_experiment(meta_props=experiment_metadata)

experiment_uid = client.repository.get_experiment_uid(experiment_details)
print(experiment_uid)
956b4a2c-67ef-47ff-acd0-de273ed61aef

List the stored experiments.

In [37]:
client.repository.list_experiments()
------------------------------------  ----------------------------  ------------------------
GUID                                  NAME                          CREATED
956b4a2c-67ef-47ff-acd0-de273ed61aef  STYLE experiment              2019-11-11T22:04:32.916Z
b4a46b63-9d47-49d3-8a9a-642e470f4fa9  MNIST experiment              2019-03-07T13:32:57.968Z
c499b36a-2e15-4846-85db-4c5d8f17122d  Distributed MNIST experiment  2019-03-07T13:24:35.372Z
------------------------------------  ----------------------------  ------------------------

Get the experiment definition details

In [38]:
details = client.repository.get_experiment_details(experiment_uid)

4. Run the experiment

Tip: To run the experiment in the background, set the optional parameter asynchronous=True (or remove it).

In [39]:
experiment_run_details = client.experiments.run(experiment_uid, asynchronous=False)

#########################################################

Running '956b4a2c-67ef-47ff-acd0-de273ed61aef' experiment

#########################################################


Experiment run uid: 6649108c-fe62-43e2-b886-fd21a6780764

0%   - Processing model-c5n237cm (1/2): experiment_state=pending, training_state=pending
0%   - Processing model-c5n237cm (1/2): experiment_state=pending, training_state=running
0%   - Processing model-c5n237cm (1/2): experiment_state=running, training_state=running
0%   - Processing model-c5n237cm (1/2): experiment_state=running, training_state=completed
50%  - Processing model-qbz9mdw1 (2/2): experiment_state=completed, training_state=completed
100% - Finished processing training runs: experiment_state=completed


--------------------------------------------------------------------
Run of '956b4a2c-67ef-47ff-acd0-de273ed61aef' finished successfully.
--------------------------------------------------------------------


As you can see, the experiment run has finished.

Get the experiment run UID.

In [40]:
experiment_run_id = client.experiments.get_run_uid(experiment_run_details)
print(experiment_run_id)
6649108c-fe62-43e2-b886-fd21a6780764

Get the run details.

The code in the following cell gets details about a particular experiment run.

In [41]:
run_details = client.experiments.get_run_details(experiment_run_id)
In [ ]:
run_details

Get the experiment run status.

Call client.experiments.get_status(run_uid) to check the experiment run status. This is useful when you run an experiment in the background.

In [43]:
status = client.experiments.get_status(experiment_run_id)
print(status)
{'submitted_at': '2019-11-11T22:04:33.953Z', 'current_at': '2019-11-11T22:04:33.953Z', 'current_iteration': 1, 'state': 'completed'}

Monitor the experiment run.

Call client.experiments.monitor_logs(run_uid) to monitor the experiment run. This method streams the training logs content to the console.

In [ ]:
client.experiments.monitor_logs(experiment_run_id)

List the training runs triggered by experiment run.

In [45]:
client.experiments.list_training_runs(experiment_run_id)
---------------  ------------------  ---------  ------------------------  ------------------------  -----------
GUID (training)  NAME                STATE      SUBMITTED                 FINISHED                  PERFORMANCE
model-c5n237cm   kandinsky - cracow  completed  2019-11-11T22:04:35.674Z  2019-11-11T22:06:52.931Z  -
model-qbz9mdw1   van gogh - cracow   completed  2019-11-11T22:04:35.942Z  2019-11-11T22:06:57.559Z  -
---------------  ------------------  ---------  ------------------------  ------------------------  -----------

As you can see, two training runs completed.

In [46]:
# List the training uids.
# training_uids = client.experiments.get_training_uids(experiment_run_details)
training_uids = [run_details['entity']['training_statuses'][i]['results']['reference']['location']['model_location'] for i in [0,1]]
print(training_uids)
['training-24lVgJ1Zg', 'training-l-_VgJJZg']

5. Results - transferred styles images

In [47]:
bucket_name = buckets[1]
bucket_obj = cos.Bucket(bucket_name)
In [48]:
transfered_images = []

for uid in training_uids:
    obj = bucket_obj.Object(uid + '/transfered_images/krakow_at_iteration_' + str(iters-1) + '.png')
    filename = 'krakow_transfered_' + str(uid) + '.jpg'
    transfered_images.append(filename)
    with open(filename, 'wb') as data:
        obj.download_fileobj(data)
    print(filename)
krakow_transfered_training-24lVgJ1Zg.jpg
krakow_transfered_training-l-_VgJJZg.jpg

Cracow

Have a look at the original picture again.

In [49]:
Image(filename=os.path.join(data_dir, 'krakow.jpg'), width=1000)
Out[49]:

Cracow + Van Gogh, Cracow + Kandinsky

Display the pictures after Van Gogh and Kandinsky styles have been applied.

In [50]:
Image(filename=transfered_images[1], width=1000)
Out[50]:
In [51]:
Image(filename=transfered_images[0], width=1000)
Out[51]:

6. Summary

You successfully completed this notebook! You learned how to use watson-machine-learning-client to run experiments. Check out our:

Citations

Author

Lukasz Cmielowski, PhD, is an Automation Architect and Data Scientist at IBM with a track record of developing enterprise-level applications that substantially increases clients' ability to turn data into actionable knowledge.

Copyright © 2018, 2019 IBM. This notebook and its source code are released under the terms of the MIT License.

Love this notebook? Don't have an account yet?
Share it with your colleagues and help them discover the power of Watson Studio! Sign Up